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ABSTRACT 

This study brings together a large body of literature 
on" indicators of changes in ^O.S, scholastic aptitude and achievement 
levels within the 194 4-1960 V^^^iod. The population target Us youth 
f rofli • pi^eschool to , post-^cq^lege graduate ages# The s^udy iS divided 
into three major sections: (1) a discussion, of the trends which have 
emerged over time that have been supported by creditable data. The 
discussion' a.pproaches the^subject by first reviewing aptitude tiest 
score' j^At terns from 1952' through 1980 on eight scholastic aptitude^ 
and admission tests. Examinees for these tests predominantly ranged 
from '1Jth grade through graduate levels* Second, an analysis is done 
on th^e scholastic achievement test score data covering 10 different 
test batteries for grades 1 through 12. Finally, section one of the 
paper coxiclude? with a briqf overview of other indicators of 
significant, though less guantifiabie changes in the target 
population during the period; (2) an annotated bibliograpny of 49 
sources ^irliich were reviewed by the author, inSiluding 34 cxtatrons 
•Trhich have been quoted' directly from Other sources; and (3) a 
relatively, complete bibli^ography 'of literature ^related to .the decline 
of test scores. This refer%ce li^t includes over 240 sources, 
(Author/GK) 
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PREFACE 

The Department of Defense (DoD), in conjunction with the Department of Labor, 
is sponsoring res€?arch to determine the aptitudej profile of the current youth population. 
Ta accomplish the project; DoD contracted with the N^itional Opinion Research CenteSr 
(NORC) of the University of Chicago to administer its enlistment test, the Armed Ser- 
vices Vocational Aptitude Battery (ASVAB), to a representative sample of youth. That 
sample was already under study in the National Longitudinal Survey of Youth Labor 
Force Behavior that NORC has been conducting over the last two years for the Depart- 
ment of Labor. 

DoD also commissioned'other studies on relevant topics which were designed to 
provide backup materials for the analyses of the aptitude profile data. This review 
of civilian aptitude and achievement test score decline over the past 20 years is intended 
to place military test score trends over the survey period into better perspective. 

\ ^ 

Tbe publication of this report as a technical memorandum provides access to this 
work by the wi^er audience of the psychological/educational research community 
beyond the Department of Defense. 
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- ■ • INTRODUCTION'. ' 

BACKGROUND ^ ; ' . 

Beginning in 19T5, the College Entrance Examination Board (CEEB), published > 
several reports detailing consistently declining Sch9lastic ^ptitude Test (SAT)^ scores 
over a'near ten-year pedod/ Whenlhe CEEB data were supported by American 
College Testing Prograpi<ACTi|^^p*^ll as an evfer-growing number of achieve-^ 
merit test score declines, educa^nal and political force^wung into action. Over 
the next thr^e years, tHe subject of declining studenj aptitudes and achievement d^- 
inated the' educational.-and psychological'^terature, with many rep(w:ts find books 7 
receiving heavy media and public exposure. 'Numerous sympo^a, commissions, and 
studies were launched to answer three key questions: 1) Were the^eclines a **real" 
national phenomonen?; 2^ Could the cause(s) for the decbW'be identified if the 
answer to 1) was yes?; and 3) What could be done about the decline?. 



STUDY OUTLINE 

♦ ' . • • ' ' » ^ ' 

This paper is intended to bring together this large body of literature on indica- 
tors of changes in U.S. scholastic aptitude and achievement l^els within the 1944-1980 
period. The population target is youth from pre-schaol to ffcst-college graduate ages. 

The study is divided, into three major sections: ^ . ' 

1) A discussion of the trends which nave enjerged over time that have, been 
supported by cr^ditaJbledata.. The discussion app^roaches the subject by first review^ 
ing aptitude test s(core~pattems from 1952 through 1^80 on eight scholastic aptitude 
' admission lests. Examinees for Ihese tests predominantly ranged from Hth grade 
graduate levels. Second, an analysis is done on the scholastic aphievement 
test scorerdata covering 10^ different tfest batteries, for grades 1 thrdugh 12. t'ihaUy, 
section /one of the papjer concludes witii a j^ri'ef overview of^other indicators of sig- 
nificanjl, though less cjiiantifiabte changes in the target nopulation during the period.^ 

7 . . . • • . 3 \ • ^ 

'\ 2) An annotated bibliography, of 49 sources which wer.e reviewed by. the 
authoA including 34 citations^ which have been qt£oted directly from other sources. 

^ ^ . ' < 

3K A relatively complete bibliography of literature related to the decline of . 
test scores. This reference lilt includes over 240 sources: 



*The author wishes to thank research assistants Saundra Watew and Marsha Wallace for inval- 
uable aid oh this study. ' ' ' ,1 

^This study does not delve into the possible causes underlying the decline in scores, except 
tangentially^ through bibliographic reference to other sources which have dealt with the numerous 
^hypotheses. * " • 
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SECTION I: EVIDENCE OF A DECLINE 

SCHOLASTIC APTITUDE f&IEASURES i 

The most clear, consistent and unambiguous evidence of the decline in the target 
population evolved in the ai>titude testing domain. Table 1 and Figure 1 provide a 
compilation of the trends in this country since early in the 1950s; these data are 
remarkably consistent. With the two exceptions of the Law School Admissions Test 
and, the Natural Science Subtest^ of the ACT, all of the d^ta.show about 1 to 3% of 
a standard deviation decline per year since the mid 1960s. ..Other major trends show 
that .verbal sqores have tended to decrease faster than qua^jtitatiye scores; that feipale* 
scores have declined' more rapidjy than male scores, particularly in the verbal domain; ' 
and that overall aptitude test scores increased from 1944 through about 1965 and 
decreased consistently through the late 1970s. The source numbers shown in Table 1 
refer to the number of the data sources in the section 3 bibliography from which 
trends have been calculated. Discussion of the individual measures of scholastic apti- ' 
tude' follows.* • . , 



Scholastic Aptitude Test (SAT) 



The SAT was introduced 55 years ago, 'with the 1941 examinees becoming the 
standardization population for tests since' (110). Test performance remained rela- . 
tively consistent through, the 1940s, followed _by a steady increase on both verbal ' 
and niathematics scales through 1964. Since 1965, however, mean scores on both * 
subtests have declined continuously to their lowest ppint in history in 1980 (199). 
Two particularly rapid periods of decline occurred; 1965-1967'and 1973-1975, with - 
the verbal score declines shaiper than the mathematics score slippage (103). The^ 
recent period, 1977-1980, shows a slowing of the, rate of decline with about 1-2 
points per year loss on each subscale with the 1980 SAT-total scores At 466 mathe- 
,matics and 424 verbal, ^e 1967-1980 SAT score decline for high and low scores 
is displaj^ed in Figure 2. ' 

Analysis ^of the .SAT changes^rfeflects no apparent geographical,* test content, 
scaledrift, socio-ecopomic^status, or racial-ethnic trends which appear td contribute 
sub^tially to the SAT score decline (103). Th6re is evidence that thfe population 
of SAT test takers has become younger, more proportionally female, and" more apt 
to be college-bound than ^|Uer year examinees (103). The variances otthe two 
subtests have remained Vfery5DHstant since the decline began, with verbal standard 
deviations .between 1966-1980 ranging from i07 to llO and mathematics from llO to 
117 (103). Thus, the expansion of the te.st taking .population to a greater percentage 
of high school seniors has not rejected a significantly wider spread of test scores<-^ ^ 



J 



1 " • 

Much of the discussion in this section reflects a synopsis of two sources: Hamischf^ger & 
Wiley (#103), H6s5, Martin, Parker, & Beck (#157). In the author's opinion, these ^e the "best", 
^ i.e., most comprehensive and objective, analyses: of the -declining test scores. 




Table 1 ' 







Scholastic Aptitude Measures 








* ■ 














Bibliography 






Time* 








Trends 


Source 
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Figure 1 — Aptitude Measures 
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Figure'2 — Proportions of College Aspirants Scorin^GOO or Higher an'd 
Less than 300 on the Scholastic Aptitude Test (SAT), 1967 - 
'^r 1980 ^ 
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Preliminary Scholastic Aptitude Test (PSAT) _ 

♦>■ - * » , - - 

, The !^§AT.is a shortened version of previous SAT forms given to high school 
juniors in October of feacih year. Siw^ 1971, the test has been cpnnectftd to the 
National Merit Scholarship Competition (NMSC). The PSAT/NMSC population tends . 
to be stronger than. the*^ SAT population and has been even more sp since the 1971 
change. College Entrance Examination Board analysis indicated -a significant scale 
drift after 1966. Thus, ^Table 1 data on the PSAT and discussion in ttiis* section . 
reflect corrected results from the ETS studies (103). PSAT verlJal and mathematics 
trends from 1959 to 1980 shaw relatively irregulaj; patterns with an average overall 
decline of about X to 2% of a standard deviation^ per year. When viewed by sex ' 
within the two subtests, however, the trends are quite- different. » Males on the ver- 
bal scale and females on the mathematics test have shown only a gradual decrease of 
less tlian 1% per^year over the period. On the other hand, females lost an average ^ 
of nearly 2% oVi standard deviation per year on the verbal scde, while*males declined 
atf^average of 1% of a standard deviatiotf^^er year an.the,fnathematics (103). Th^e 
two subgroups thus accounted for vir^ally aU of ttie PBAT declines oyer the is year 
^ period. The reasons underlying these changes amjiot^^clear, although the variances of^ 
the mathematics scoses.are sharply hi^er for both males and females over the peri6d 
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while the verbal score variances are relatively level (103). These results would suggest that 
the PSAT later populations were more heterogeneous in mathematical aptitudes^ but not 
verbal aptitudes. As this significant subtest X sex interaction effect did not evolve as 
strongly in any airthe other aptitude measures, the author concludes that it likely is a 
• result of unspecufied characteristics of the PSAT examinee pool. 

♦ 

— American College Testing Program (ACT) 

The ACT consists, of a battery *of four college admissions tests: English, Mathe- 
' matics, Social Studies, and Natural Sciences- The battery is directly ^ based upon the 
V- Iowa Tests of Educational Development ^ITED). Thus, the test might be considered 
niore achievement oriented than the SAT/PSAT just discussed^(103). The ACT also 
yields ar composite score whicH is an average of the four subtest scores. ACT composite 
V score trends over the 1964-1980 period are basically v^ similar to the combined SAT 
verbal/ mathematics trends during the same time, frame. On k per year basis, the average 
ACT composite decline has been about 2%»of a standard deviation (157), with about 
double the decline ii^ female mean scores as compared with male scores .(102) over the 
period. Within ,the four subtests, female declines have been greater in the English and 
Social Studies tests, while the males have been greater on the Social Studies, Mathe- 
matics tests. Relatively little change occurred on the natural sciences subtest, although * 
m^l^ tended to improve slightly over the .period and females remained practically 
unchanged over the V7 -y.e&r timeframe. Munday, in a comprehensive analysis of AQT 
score decline', coricluded that examinee grade level and geographical region variabley^^ 
did not account for score declines. However, he pointed out that the increasing pro- 
portion of females taking the test (45% in 1965 vs. 52% in 1974) and increasing vari-" 
ances of scores on all subtests were likely correlates with J;he declining trends (157). 

Minnesota Scholastic Aptitude Test (MSAT) 

Over 90% of Minnesota juniors took the MSAT, a test*given from 1958 through 
, 1973. A form change occurred, in 1966-67,- thus data in Table 1 reflect both pre 
apd post change trends. In general, means and standard deviations increased from 1958 
. through 1966 and both steadily deareased until 1973 (103). No breakout by sex was 
' available for this study, although the 2% standard deviation per year decline in the 
later period was similar, to SAT? and -ACT overall results. ' " 



Graduate Record Examination (GRE) 

The'GJlE is the most popular of the graduate admission tests. It was^^eveloped 
by Educational Testing Service and yields both verbal and quantitative scores. The 
average verbal and quantitative declines from 1967-1980 were about 1% of a standard, 
deviation . . . similar to SAT, PSAT, MSAT and ACT rates. Again, nt) sex by sub- 
test data were available for the GRE^5). ^ 

' tJ#6raSuate Mana^ment ^missions Test,(GMAT) • ^ 

GMAT test results are used for selection to graduate schools of business. The 
, 1967-1975 trend showed an average of 1 1/2% of a standard deviation per year decline 
over the eight year period (95)."' - ' , 



Medical College Admissions Test (MCAJ) 

, MCAT results over the 1967-80 time j^eriod reflect similar trends for verbal 
and quantitative subtests. The verbal test mean decreased an average drop of about 
2% of a standard^ deviation per year. However, the MCAT quantitative mean fell 
less than 1% per year over the period (95). ' 

Law School Admissions Test (LSAT) 

/ 

Like the MCAT-Q, the LSAT mean score increased over the 1967-1975 period, 
an average of 0.6% of a standard deviation per year (95). 

Stanford-Binet . ' ^ 

Arguments over the classification of tests between aptitude and intelligence. (IQ) ^ 
rage within the literature, and it is not the purpose of this paper to enter this fray. 
The Stanford-Binet, an *1Q" test, is an individually administered, highly reliable measure 
which has been administered nationally, since early in this century. Thomdike reviewed 
Stanford-Binet trends over a 70 year period (in 103, page 4).* He presented data to 
^hom that preschoolers of 1972 averaged 10 points higher than their age-mates of the 
tl9^; that high schoolers averaged 6 points higher; and that 4th graders averaged 3 
points higher. He also showed an average 3 point drop for preschoolers between 1972 
and 1975. This one source was the only reference located for this paper which analyzed 
IQ test score trends over time. A much more complete revie^^ in this area should be 
done. One remarkable source for this further examinatioif would be Shuey's The 
Testing of Negro Intelligence, Second, Edition, which was published in 1966. This 
extraordinary book provides data from over 600 studies ot IQ and aptitude over a 
period of over 50 years. The e^b^of combining Shuey's extensive tables into a com- 
prehensible analysis over time was beyond the scope of this paper, but the effort should 
be made. It is difficult tot the author to understand why the extensive analj^sis of the • 
declining test scores which occurred in the 1975-1978 time frame virtually ignored 
the individually and group administered IQ test score trends. Although the precise' 
relationship betweea what IQ tests measure as compared with what aptitude and 
achievement tests measure -is unclear, the trends from the foamer should prove useful 
in the sQialysis of the lattei;. 

SUMMARY OF APTITUDE TESTING TRENdS 

The aptitude testing data covering the entire period from the early 1960s to 
1980 show remarkable consistency. With the exception of slight increases on the MCAT-0 
and LSAT, the other measures of scholastic aptitude of youlih reviewed have consistently 
decreased at a rate of about 1 to 3% of a standard deviation per year. This ttb^id con- 
tinues today, although there is some evidence that the rate of decline has lessened 



* Thorndike, R:L. Mr. Binet's Test 70 Years Later. Educational Researcher, 4, 1975, 3-7. 



somewhat in the past three years. And, although the declines are clearly related, to 
some degree, to a changing population of test takers, particularly women, the pattern 
has remained fairly consistent over breakouts by race-ethnic group; geographical arfeas; 
age beyond about 10 years old; coTitertt ar^; and socio-econpmlc status. Thus, the 
"causes" of the consistent aptitude test score patterns are n8t at all clear. The general 
conclusion of most authors on the subject is that there is little doubt that there are 
multiple factors which have lad to the trend* 

! • ' 

SCHOLASHC ACHfEVEMEjJjT MEAiSURES \ 

Table 2 and Figure 3 di^ict 1964-1980 mean results for 10 achievement test 
.'batteries.' ''Data for the indijfdual batteries have been grouped, when available, into 
grades.X-4, 5-8, and 9-12 arm by subtests that roughly parallel the verbal/quantitative/ 
composite breakouts of the^ptitude me,asttres. As might be expected, long-term trends 
across achievement content areas are not as consistent as across the more "factorially 
pure^' aptitude areas. Within table 2, trend data are displayed, by percent cliange in 
standard deviation per year where both means and variances were provided in 'the origi- 
nal source. . ^ 



Stanford-Achievement Test 

For the 1964 to 1973 period, Copperman (48) reported that mean Stanford- 
Achievement Test scbres in reading and mathematics computation.decreased consistently 
the older the test group from grades 1 to 8. He reported that Metropolitan Readiness 
Tests indicated that 1978 preschool children far exceeded mid-1960's preschooLchildren 
academically and that the^arly scholastic improvement held through 2nd grade. There- 
after, diminishing performance at an increasing rate evolved to the point wherein 

. . . eighth grade students in 1973 leak about as well as 
seventh graders in 1964, and compute about as well as 
sixth graders at that time. (48) 

The overall rate of achieVemfht loss was approximately twice as fast for math com- 
*putation as for reading on the average. • , ' 

Mejtfopolitan Achievemefit Test (MAT) 

A renonning study *which looked at MAT score change* between 1970 and 1978 
showed .a siriailar pattern for childr&n from 2nd through l(ftn grades^ Student achieve- 
ment increased in the jBrst *i3 years, ^d drbpped sharply from 5th through lOtJi grades. 
The pattern on the MAT was consistent across 5 subtests, reading, language, math, 
science, and soiiial studies, with the greatest declines in the latter three content areas. 
The average declines at the eighth grade level ranged from 2% standard deviation/year 
in language to nearly 5% standard deviation/year in social studies (48). By the tenth 
grade, average subtest dec^ifec reflected a 4.5% "standard deviation per year decrease . 
in reading, an over 6% stan^ara deviation per year in math, a 4% standard deviation 
per year decline ifi language, aVear 7% standard deviation per year decrease in^sgience, 
and an over 6% standard deviatlpn per year decline in sotSal studijyerformance (47). 
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• * Table 2 • 
Scholastic Achievement Measures 





c 

Instruments / 


Time 
Penod 


GradeTj} 


Annual N 


4 

Ar6ai 


Trends 


BibhograpHy 
Source 




Stanford Achievement 
.Test 


1964-1973 


1-4 

- 5-8 


6.000.000 


National 


+1,3 ' 
-Z.1 


47.48 




Metropolitan Achievement 
Test IMAT) 


1970-1978 


24 
9-10 


400^(100 


Natior^l* 


. " +0,6 
-2,8 


47 




^ California Achievement 
Test (CAT) * - 

» 


1970-19Z8 


2 
•5/8 

11 


Not 
Available , 


National 


"Slight Gam" 
Rda -0,3 
Mth -0.4 

Rdg -2,0 
Mth -35 


' 47 




io^^a Tests of Basic 
SRiIIsIiTBS) 


1965-19^0 


3-4 
5-8 


50.000 


Iowa 


I Rdg +1 1 
Mth +0,2 

rnmrT^ +10 

Rdg -2 0 
Mth -2 9 
- Comp '1.4 


• T57. 119 




Comprehensive Tests of 
Basic Skills (CTBSI 


•19^-1975 


2-4* 

5-8 

9-10 


' 200.odo 


Natignal 


C6mp -0,T 
Cornp -1 2 
Comp. -3,0 


loy 




Iowa Tests of 

Educational Development ,' 
OTEO) 


1965-1980 


9-10 
1M2 


Not 
Available 


Iowa 


Voc -0,4 
.Quan -1,2 
Comp -K6 

, Voc -0,9 
Ouan -1,2 
Comp -1.5 





National Assessment of 
Educational Progress 
(NAEP) 



Rdg 
Wri 

Mth 
Sci 



1971-1980 
1969-1^74 

1973-1978 
T970-1979 



75.000 to 
100.000 



National 



fldg 
Wn 
,Mth 
tci 



ylc 



Rdg 
Wn 

th. 
'Sci 



12 



Rdg 
•Wn 
Mth 
Sci 



+,06 
H) 
-0.7 
-0.9 

+o.r' 
-u 

-0.8 

-1.1 

-0.1 
-1.3 
-1,4 
-2.9 



53,91.115. 
163. 1JS4. 165 
*166. 167. 166: 
235 



Canadian Tests of 
Basic Skiiis 




1966-1973 


3 ^ 
8 


Available 


Cani^ 


Comp -0,6 
Comp -1,0 


157 


s 


Iowa Sileht^ Reading 
Tests^ {ISRTI 


■ # 


1944/1976 


6 

Id 


15.000/B.OOO 
11,000/8.000 


Indiana 


+1,8 
+0.8 


79 




General Educational 
Development (GED) 




{964-1979 


Meari 

ia 


120.000 to \ 
700.000 « 


National 


-0.86 % 
Met Std$/Yr 
{73% Vs. 60.1%) 


92 





*ISRT score* adjusted for exammee age changes between testing sessioris, 

\ ^ 12 



I 



r> 




+3% 



♦ 1% 



NO CHANGE 0 



7 



O 

< 
<r 
u 
z 

s 



-2\ 



-3% 



Sunlord 
Achievement 
Test 



1964 • 
1973 



Meuopohun 
Achttvermnt 
Test 
{MAT) 




1970 • 
1978 



CaltforriM 
Achievement 
Te$t 
(CAT! 



S 2' 

GC GC 

0 ^ 

Q t 

GC S 

1 I 



197Q . 
1978 



Iowa Tests of 

Bdsi;C Skills 
OTBS) 




1965 . 
1980 



Compfehensive 

Tests of 
* Baste Skills' 
(CTBS) 



1968 . 
1973 



lovwa Tests of 
Educational 
Development 
UTEOI 




1965-^ 
1980 



National Asseument of 
Educational Progress 
(NAEP) 



o „ 
o 

, Q 



1 ^ 



- oc o 



o 
I o 

z 

I- 
s 
I 



1971 . 
1980 



1969 • 
1974 



1973 . 
1978 



1970 . 
1979 



Canadian 
Testspjr 
Basic Sgfni 



W. flit 7— 



: ■ ■ 



1966 • 
1973 



Iowa 

Siltnt 
Reading . 

Tests 
(ISRT) 



1944/ 
1976 



*Mean proportion of a standard deviation per year." 



Figure 3 — Ac|pievement Measures 
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California Achievement Test (CAT) 

Reviewed data on the CAT are relatively sparse. The CAT pattern is very sfflaUar 
to Stanford and MAT patterns with slight reading and math increases ovef*theM970-78' 
period' at the 2nd grade level, slight decreases at grade 5, and accelerating>^creases, 
through the 11th grade averaging 2 to 4% per year of a standard deviation. 

Iowa Tests of Basic Skills (ITBS) 

The ITBS battery yields grade 3-8 achievement scores in five major areas (vocabu- 
lary, reading, language skills, work-study skills, and mathematics skills) plus a composite 
score. As documented by Munday (157), ITBS scores on all six scales at all six grade 
^ levels increased consistently from 1955-1965. During the 1965*1980 period, scores 
stabilized for 3rd and 4th graders on all but capitalization and punctuation subscales 
of the language skills test, which declined rather 'dramatically over the latter period. 
For the 5th througH 8th grades, average scores declined ^t $n increasing rate with each 
school year, particularly in the vocabulary, reading, and language skills areas. The ITBS 
trend for 5th through 8th graders on the percent standard* deviatiorrper year scale was 
about 2% decline per year in reading achievement' and a 3% decline^er year in math. 
Thus, ITBS data reflect very similar results to the earlier discussed achievement tests. 

Canadian Tests of Basic Skills • i ^ 

The Canadian^ Tests of Basic Skills are similar to the ITBS and are widely-<used^ for 
Canadian elementary school children. 1966-1973, score trend analysis showed: ^ 

Difference^ 'on the composite range from 1.5 grade ^equivalent 
months inlgrade 3 to 6.1 grade equivalent (months) in grade 8. 
I ' Differences were greatest in the four tests of language skills,^ 
capitalization in particular ... (157) • , 

Comprehensive T^sts of Basic Skills (CTBS) 

The CTBS battery tests children from 2nd through lOth grades, yielding scores on 
matfeematic application, concepts, and computation; reading, vocabulary, and compre- 
hension; language mechanics, spellmgr-fiaid^pression, and rq^ence skills. Comparison 
of 1968 and 1973. nonning data show practically unchanged performance on all three 
major content areas through 5th or 5th grades and worsening performance through 10th ^ 
grade at an incrementally widening rate. By the 10th grade, the average 1973 composite 
score was 35 scale'^score points lower on a scale ranging from 250 at 2nd grade to 620 
at 10th gra^ie (103). 

Iowa Tests of Educational Development (ITED) 

^ The ITED battery tests high school students in seven achievement areas: 
Expression, Quantitative Thinking, Social Studies, Natural Sciences, Literature, 



Vocabulary, and Use of Solirces. Performance patterns aire simQar to SAT and AQIX 
trends. In general,^ average performance across all subtests increased through the mid- 
1960s atid has decreased since, most profoundly in th*e literature and quantitative 
scales for 'the 12th grade. Natural Science scopes appear to have stabilized around" 
1971 at all fovur grades, and remained fairly constant (177). " 



National Assessment of Educational Progress (NAEP) 



• The NAEP is a federally-funded project designed to measure national scholastic 
achievement across time at the 9, IS, 17 and young adult (26-35) ages. Starting* in- 
1969, assessments have been made in 10 curpcular areas .which have been repeated 
in four year cycles. The tested areas are: art, career and occupational development, 
citizenship, literature, matiiematics, music, readinjj, science, social studies, and writing. 
As Table 2 shows, the general pattern reflected in the applicable NAE^* subtests is 
similar to those reflected in the commerically^produced scholastic achievement meas- 
ures, except on the reading scale, where little change occurred from 1971 to L980. 
Hamischfeger and Wiley (53) suggest that the NAEP reading scale ^contradictioir with 
other verbally-related scales is a function of the NAEP test contents and tasks. They 
report tha^t: * . ^ 



NAEP's items address basics far below the 11th and- 12th grade 
curricula, while the oth^ tests are oriented towards more grade- 
level specific content. . . . The NAEP increasa . . may not 
ultimately contradict the findings of decline in more comfriex 
content (53). , ^ 

A$ shown in Table 2, the mathy.5cierice, and writing subtests of the NAEP havi 
duced results very similar to. other measures reported, with accelerating decrea; 
performance with educational level. 



ro- 



General Educational Development (GED) 

The <3ED is an achievement battery used to determine qualification for an exami- 
nee ip be issued a credential ''equivalent" to a high school diploma. The battery is 
produced by the GED Testing Service of the American Council on Education. It is 
composed of five subtests: The writing skills test, the social studies test, the science 
testy the reading skills test, akid the mathematics test. GED data acioss the 1952-1979 
time period have been reported (92) by. percent of examinees meeting standards for the 
credential. It should be noted that the population of examinees has changed rkther 
drastically over the period: total number testefl increased jfrom about 30,000 in 195g 
to 774,000 in 1979; the average age of examinees has declined jfrom near 30 years old ^. 
in the late 1950s to aroimd 25 in the late 1970s; and the proportion of veterans to - 
non-veterans has gone from .6 in 1953 to less than .3 by 1966. While accepting the '"^^ 
'I impact of the changing demographics of the testing population, an analysis jof, the 
pattern of sUccessf^il GED applicants is of direct relevance to the present study. In 
1954; 80% of GED examinees met minimum standards. The proportion passing 
declined very slowly through 1962 (75%), but has dropped to under 6Q% for 1978-1979. 
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Iowa Silent Reading Tests (ISRT). 

• ' ^ ' \. ' - ' . 

One final large scale study was reviewed. In 1976 Roger Farr and his associate^ 
(71, 72, and 73) at Indiana replicated two 1944-45 statewide reading achievement* 
tests for 6th "^and 10th graders. The researchers found relatively small decreases in per- 
formance over the 32' year period, however the 1944 year-group was approximately 10 
months younger ior 6th, graders bM, 14 months younger for TOth graders. Farr adju'sted 
his test score, data to reflect the age differences and conclude^ that *^e assumption, that 
the reading abilities of students is declining is unsupported by this study" (79). It should 
be^noted that Farr's study compared only two data points in time, 1944 apd 1976, and 
thus us Hot comparable with the ^great majority of the pther studies iti this review which 
gener^y cover the later 1960s and 1970s. It is well documented that test scores in 
general rose until the niid 1960s when the* decline began to be extensively reported. 
Although pure cQnjecture,. it is v^ry possible that if data comparable to, Farr's were Avail- 
able for the mid-1960s, that mean scores might have been considerably abo?e the 1976, 
scores. » Presumably this conjecture will never be either* supported or rejected. ^ 

Summary of Achievement Testing Trends ' 

In general, the author found consistent evidence of achievement test score, declines, 
in all areas tested above grade 4, for the 1960s through 19i70§. Pre-school academic 
preparation and early (lst-3rd grade) children generally- scored higher on all measures, ^ 
with 4th grade students fairly constant. In the author's. opinion, these trends are real, 
national in scope, and continuing, though at a deof^asing rate of decline since about 1977. 



Other Indicators of Population Performancfe Change 

■ . * ' ^ ^ ' 

The literature on the test score declines includes many references to other indicators 

of a national declining level of competence of youth beyond simply test performance. 
Among others, these indicators(!include elementary ,''|iigh school; and college teachers' 
t opinions, statewide competency-based assessment, measures of curricula content at 
aU levefe, analyses of classroom hour^ attendance per stUdeUt, analyses of teacher educa- 
tion and practices, and physiological hypotheses about diet*, drug, medication, nuclear 
radiation and other^pos^ble correlates of declinii^test scores. ThejSnnotated and full 
bibliographies of this, review cite these and ♦other reports and studies. It is beyond the 
scope of this review to attempt to analyze the projDaJ?le (possible) causes of the declin- 
ing scores. . e ^ ^ / . . 

SUMMARY OF OVERALL TESTING TRENDS * ^ } ' * ; 

It is evident that natiorihl youth performance on scholastic aptitude and achieve- 
ment testes have been in a state of decline. Assuming comparability of population, for 
ciurent military enlistment eligible youth (17-24), the scope of the decline based upon 
this literature, review of civilian test score change would likely represent a general decrease 
of about onfe-fourth to one-third of a standard cfeviation on the average fronx the 1970 
pool of AFQT examinees, or about 2-3% of a standard deviation per year. This rate 
would equate to a decline of approximately 4-5 AFQT raw score points for the average 
Military Enlisted Accession from 1971 to 1980. * / ' 

X 
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SECTION II: ANNOTATED SELECTED BIBLIOGRAPHYl ^ . 

« 

Advisory Panel on Scholastic Aptitude Wst Score Decline, On Further 
Examination, 1977. o 'r- 

This report, known popularly afthe Wirtz report afte? Willard Wirtz, the 
' advisory panel chairman, covers an extensive r^ew sponsored by the College 
Entrance Examination Board- (CEEB) to investigate the SAT decline.. 
The blue-ribbon panel of 21 authorities was formed in late 1975 and the 
report was publish'^ in July, 1977 as a -separate CEEB publication as well 
as testimony to the 19.79 Examination of Basic Skills conducted by the 
Sub-committee on Education, Arts, and^ Humanities of the Committee on 
Labor and Human JJfesources of the U.S. Senate (#73). 

The Wirtz report is. an , excellent, well thought out treatise ^pH the SAT 
' decline. Besides the basic document, CEEB .has published 27 appendices * 
which represent individual studies and reports on related topics commissioned 
specificaUy by' the panel. The 27 appendices a^ejhstracted in 2|n annotated 
bibliography to the main report and were publisMB as a separate CEEB 
publication. 

The report itself is divided into the followmg sections: ' 

Part One: The Scholastic Aptitude Test and th'fe Test Score Decline. 
Part Two: Arj. Unchanging Standard. , 
Part Three: The Two Test Score Declines. 

' Pact Four: Circumstantial Evidence. ' ^' 

Part Five: Summing Up. 
Annotated List of Studies and Papers. 

Bibliography. ^ . . 




* # 16. Arnold, C.B., Could There Be a Medical Basis for the DecFining SAT 
Scores'? J^uary 1977. ' ^ ' 

A brief summary anh ^alysis of the* medical and epidemological literature- 
dealing with nutrition, drugs, genetic and prenatal conditions, labor and 
delivery, child developmerlt, health status, and health care over several 
decades in the United States. It concludes that the decline in SAT:.3qores 
is probably not thfe result of disease processes or physical environmeritisll 
factors. 

* * A^terisk^ entries in this annotated bibliography indicate quoted citations from the following - 
sources: . ^ ' ^ 

♦From Source #5. ^ " 

♦*From Original Authors* abstracts. • ^7 < . 

* ♦♦♦Froih Educational TechmlQgy, November 197^5 
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20. ?aird, L., & Fiester, W., Grading Standards: The Relation of Changes in , 
Average Student Ability to the Average Grades Aw^xded* (ED 054 226, 
1977)^cbkQerns trends in faculty grading practices. < 

"^/^ Authors Bairdand Fiester studied the meahSi^nd standard deviations of^ 

I freshmen classe3 on^^ ability test, along with*the means and standard 
deviations cJf freshman-year grades over a five-year period, which we/e* 
available from several hundred colleges. ^ ' ^ 

Based on analyses of the data, they observe: f ' • ^ 

(!)• Within^ any given year colleges whose incoming students were 

bright tended to award higher grades, but tljere was considerable 

room for variance from this trend. - ' 

(2) -The average ability levels of colleges* change very little, even over 
" < five periods, [and] 

* . (3) When* the average .abilify ol students' incre^^e^ 
. about the same levels of grades are awarded. 

^ ^\ The paper discusses implications of the observed grading practices for 

college admissions and counseling, ajjd raises questions about the quality of 
students' wojdTv^s! aptitude and student attitudes toward the competitive 
gradi^jg system. The authors conclude: 

In sum, the present results provide little evidence that^ faculties will 
adjust grades to correspond to changes in average student ability 
over some time lapse, . . . [that] permanent or absolute standards ^ 
are employed in evaluation or that faculty generally take the changing 
' ability ^evel of their student bodies into account^. . . We n^ed to 
devote as much attention to faculty gi^de giving behavior as we have 
to student grade" getting behavior. * 

# 23. Beaton, A.E., Hilton, T.L.', & Schrader, 'W.B. Changes in the Verbal 

Abilities of High School Seniors, College Entrants, and SAT Candidates 
Betwee)i 1960 and 1972, January 1977. " ^ 

Two maj^r 'social science research efforts — Project Talent in 1960 and the 
National Longitudinal Study in 1972 — obtained extensive data on ability 
, and other significant characteristics for national probability s^ples of high 

School seniors. Both surveys made follow-up studies to detennine which 
■*of%ese seniors had entered college in the year following high school 
graduation. This study was designed to use these two exceptional* data 
bases to document changes between 1960 and 1972 for high school 
seniors, college entrants, and the SAT-taking population. The data of the 
national surveys were supplemented by the results of a special equating 
study and by a search of SAT files for about 20,000 members of the 
Talent sample. ' i ^ 

The study was concerned with three groups — high school seniors, college 
entrants, and SAT takers. Because reading scores were £^vailable for all 
three groups,, it was*|)ossible (after equating the reading tests used )• to 
measure c^xangei^ reading ability for all three groups; In addition, | 
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it was possible to study subgroups of the three rriayi groups. The sub- 
groups were detined on the basis of each of the following characteristics: 
age, sex, father's education^ mother's education, father's occupation, 
nj^other's occupation^ fanlily configuration, high school' curriculuni^ and 
expected college yiajor field. ' ^ ^ 

The main conclusions of the study are as follows. * , 

1. All three groups showed a decline in reading ability between 
1960 and 1972. The decline of SAT takers, however, is markedly - 
greater than for the. other two groups. , 

2. A much greater increase in ttie proportion of low-ahility than of 

. high-ability students who took the SAT appears to be the predomi- 
nant source of the SAT score decline between 1960 and 1972*^ 
The decrease .in ability level of^high school seniors ^uring this 
period a lso contributed to the decline. ^ . . 

3. There is some evidence that SAT scores earned in 19^ andT972~are 
not precisely comparable. The data suggest that the 'actual decline 
in average verbal ability of SAT candidates from 1960 to- 1972 was 

, ^ ' somewhat greater than the SAT scores indicate. 

4. There were appreciable changes in the background characteristics 
studied for all tiiree groups. None of them mide a major con- 

. tribution to accounting for the score decline among high school 
seniors or college entrants. A decrease in the percentage^f SAT • 
takers entering four-year colleges may help to explain the greater 
score decline observed for this group. ^ 



Braswell, J. & Petersen, N., An^ Investigation of Item Obsolescence in the 
Scholastic Aptitude Test October 1976 (revised January 1977). 

Two panels, on% concentrating on the verbal sections ofjhe SAT and the 
' other, on mathematical, were appointed t^) revie\7.and*fate questions that 
appear^ in earlier editions of the test and then in more recent editions. 
Raters were asked to indicate how the difficulty of each question might 
be expected to change between the two adtoinistration dates. The raters' 
t predictions were then compared with available item analysis data. "For. . 
the most part, the changes predicted by raters were not substantiated by 
the statistical agalysis. While the relative difficulty of some questions 
changed between administration .dates, it'jwas not possible, except for a 
few mathematical questions, to attribute these changes to cuirricular change 
or to broader social factors. These mathematical questions were predicted 
by the raters to be relatively easier at the more^cent administration, and <> 
their prediction was supported' by the statistical analysis!* 
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TTie Te^t Score ^?ec/^^lg, Parts One and Two, in two successive issues of Educational 
TeQhnology.<r June and' July 1976. . , 

These,two Issues of Edacationai Technology brought together 15 articles on selected ^ 
aspects of declining test scores. The articles dnd their authors follow: v 

Part One . ' - - * - <^ 

^r- t?l06. HamiscKfege?:, A.,"^ Wiley, ,D., The Marrow o,f Achievement Test Score Declines. 

#107. Harris, W., The SAT^core Decline: Facts, Figures and Emotions. 

# 82. Ferguson, R^The Decline in ACT Test Scores: .What'Does It Mean? 

■IP 86. Forbes, R., Assessing Educational 'Attainments. 



#193. Rippey, R., The Test Sc^e DecTuieT^f-You^^o^ KnawJA^ 

Going, How Do You Expect to Get There? *^ - ' ^ 

# 36. Burns, R., Minorities, Instructional Objectives and the SAT. 

#239. Walberg, H., Changing IQ and Family Context. 



198. Sfitpone, C.V., & Giuliano, J.R., The Tert Score Decline: Are the^^Public 
Schools the Scapegoat? , 



Part Two ^ ' * 

#129. Kapfer, P., Kapfer, M., & Woodruff, A., Declining Test' Scores: Interpretations,* 
Issues and Relationship, to Life-,Based Educatiop. 

# ^80. "Feldhusen, J., ^Hynes, K., & Am&, C, Is a Lack of Instructional Validity 

Contributing to the Decline of Achievement Test Scores? , > • 

#237. Throne, J.M., Has the Key to.the Mystery of Drops in Standardized Tesif 
Scores Been 'Discovered? 

# 40. Champagne, D., & Robferts, E>An Exa:qise in Freedom: A Place Where ^ 

Test Scores Appear to Be Rising. ' . /" 

, # 3. Acland, H., If Reading Scores Are Irrelevant, Do We Have Anything Better^ 

4_ 56-. Della-Piana, G.,.Odell, L., Cooper, C.,-& Endo, G.* The Writing Skills 
Decline: So What? ' " ' > 

» * 

The 15 articles present a picture of various lines of thought on- the test score 
declines,"^ though as a group there kre little data provided. Many of the sam« • 
authors have better documented publications outside of the two-issue Edu- 
cational Technology series. ' 
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Breland, H.M.,'Fami7y Configuration Effects' and the Decline in College 
* Admissions Test Scores: A Review of the Zajonc Hypothesis. September 
1976 (revised February 1977). 

^ o ' * . 

A hypothesis that part of the SAT score decline is a result of changing 
American family sizes and configuration is explored. This possible explana- 
tion of declining SAT scores had been offered by Robert B. Zajonc- in an 
article in Science that reviewed the evidence for a relation between family 
configuration and cognitive development. Since a number of investigations 
have shown that **early-bom" students.— those who were the first or second 
child in their families — and members of small families tend to have higher 
scores on tests such as the SAT, a change in the representation of students 
who are earlybom and members of small faftiilies in the population could 
possibly cause a decline in the average for thfe total population. The 
hypothesis is explored through a consideration of the m^ituHe of the 
change in family configujrations over the years of interest ""and the magni- 
tude of observed score differences for the SAT. It is concluded that, 
while the Zajonc hypothesis seems sound, it could only account for a 
small portion of the total SAT score decline. ^ 

Breland, H.M., ThS SAT Score Decline: A Summary of Related Research. 
January 1976. 

This paper surveys the available evidence pertaining to the scor^ decline 
in* tOTnsjrf fivyiyppti^^ 
coiTege-boimd population, the schools, and societal factors. It contains a 
description of the SAT scbre decline in both graphic and tabular form as 
well as population data for the years 1957 through 1973. Based on the 
d^ta available at the time the panel beg;an*its deliberations, it is concluded 
that the evidence confirms a general decline in the abilities of the college- 
bound population but is less clear with respect to the h^ school popu- 
lation. It is speculated that the declines in the college-bound population 
are due to several factors.acting in combination: increases in the proportion 
of low- and middle-income students, reductions in the numbers of test 
repeaters, changes in the mix of SAT-user colleges, sgid less-strict policies 
regarding college admissions. • \ 

\ 

Chall, J.S., with Conard, S.Sr, and Harris, S.H.' An Analysis of Textbooks in 
Relation to Declining SAT Scores. November 1976 (revised March 1977). 

The authors analyzed the reading selections from six SATs, two from the ' 
stable years (1947 and 1955), one from a pivotal year (1962), and three 
from the declining years 1967, 1972, 1975). Samples of the textbooks ' 
used most widely by these six SAT cohorts during their ^elementary and 
high school years were analyzed — 35 textbooks and ap^iroximately 20^ 
workbooks and teacher*s guideboojcs in reading, grammar and composition, 
•literature, and history for grades 1, 6, and 11. , 
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• The various textbooks and SAT reading passages were analyzed lising various 
indices of difficulty and challenge the Dale-Chall. Readability Formula for • 
level of reading/comprehension difficulty, Chall's Reading Stages for level 
of linguistic and cognitive maturity, a rating scale for assessing question 
difficulty adapted from Bloom's Taxonomy, etc. , 

1. Readability analyses of the SAT passages reveals a general 
decreasing trend in difficulty from the two stable'years (1947 and 

' 1955) through the pivotal year (1962), to the declining years 
(1967, 1972, and 1975). 

2. A possible gap between the difficulty of the SAT passages and the 
difficulty of the llth-grade textbooks is indicated. The SAT 
passages proved to be the most difficult of any -of the materials 
analyzed — more difficult than any of the textbooks. Two of. ^ 
the' SAT passages were on 'the level of grades 13-15 and four on 
the level of grades 11.12: The history, literature, and ^ammar 
and compositiort llth-grade textbooks had an average reada- 
bility level of ^ades 9-10. ^ . , ' 

3. There are signs in the data of a reqent Increasing challenge in 
the textbooks, particularly at the elementary level. 

4. The authors find what appears to be a particularly low level of 
challenge in writing. Generally the assignments in reading, history, 
and literature textbooks ask mostly for underlining, circling, / 
and filling in of single words. Few assignments ask students to . 

write a-paragraphrstory,-letter, oriheme. 
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* # 42. Qeary, T.A., and McCandless, S.A. Sumrnary^ of Score Changes (m Other , 

Tests). February t, 1977. An excerpt from the authors' "6core Declined 

and Grade Inflation," a paper delivered at the annual meeting of the , 

Northwest Association of Schools and Colleges, Portland, Oregon, Decemhfer 1976. 

Summarizes the experience of other testing programs, indicating the degrees 
to which average scores in each changed 'over the period of the SAT score 
decline. 

0 

* # 58. Donlon, T?., and EchtemacKt, G/" A Feasibility Study of the SAT Perform- 

ance of High-Ability Students from 1960 to 1974 (Valedictorian Study). 
October 1976 (revised Febniary 1977). 

This is a study of the SAT records of valedictorians and salutatoriaAis in 
' three groups of schools: "experimental" schools, selected for stability 

but by less sjxingent criteria; and "private" schools. For each group. 
SAT-V and SAT-M scores in the even-numbered years from 1960 to 1974 
. were studied. . ' ' 

For . the ''experimental'' group, there is no evidence of score>decline in 
either SAT-V or M. Ipstead the trends have been toward very modest 
annual increases. For the "comparison'' group 'the data indicated an 
initial period of increase, followed by a decrease\ Each of these twp 
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special sample differed significantly from the SAT-taking population. The 
''private" schools, -however, Were j;iot significantly dijfferent, shpwing a 
\ decline simil^r^Jto^ the to eblI^^'']^oar4 population;:' 



6Q. Ebel, R., Declining Scores: A Conservative Explanation, Phi Delta Kappan, 
Xrecember*1976* ' ^ ' ' - 



\ y :Ebel ijuestionsjtlie VRousseauean p)ailpsophy" which .has taken over edu- 
cation in recent decades. He calls for a ^*back to basics," and asks^ what 
' ^ Ithe mi^on.of.scl^ools is. He concludes by defending the content of testing 
anti-ca^g for, better ii^ste]acti^ in the cognitive 'areas tapped by the tests. 
. \ v^^No^data^Tarticle is esserttially a logical argument. • 

* 62. .EcKteJfpacht, G* A Comparative Study of Secondary Schodls with Different 
^^ ^ ' ^66re Pattens. October Jj^6 (revis^ January 1977).- 

w " In this study, the curriculum, mstitutional, teacher, and student factors 
5^ associated with those schools having Isirge decreas^ in SAT score averages^ 

' ^ .^^.^^ J were compared with the same factors^associated^^^IQi schools having < 
increasing or steady SAT score averages.' It wp believed that by identi- 
^ fyi^g these factors, some insight into the role/ that school characteristics 
have played in the score decline might ctAiie light. Although some 
; significant differences between the two groups'were found in terms of . . 

, w • enrollments in acader^c courses, ability grouping, age of schools, dropout 

\' rate, and teacher experience, these were judged not to explain a large 

-M? ^ ; - portion of the decline. Indeed, differences among schools do not appear 

^' . to have had large differential effects on the decline of scores. 



- ^^'^^ Education Commission of the States, heading iw America: A Perspective 

' ^^^-jS . on Two Assessrfients. October^ 1976. . , ^ 



^ The report presents data on changes in social stt^dies' achievement for young 

^" '\ , ^ ' Americans a^ed 9, 13, and 1^ .during ihe p"^ods 1972-76.*. Data were ^ 
1^^.^ / , ^^^^ f t ^ collected tiirpugh two surveys conducted by the Rational ^Assessment of 
^ C J ;;^. < Educifeonaf Progress ;iWing the 1971-72 and' 1975-76 ^choor^ears. . 

^ i-f, T^ items were categorized in three major areaffr knowle<Jge, skills, and 

attitudes* Four tbpics included amder Imowledge were economics, geography, 
^ ' « -\ '^^^-^^ 4^ - ^ Jiistory,^d politics. Skill items diea^t with ability to obtain and interpret 
^ ' "t^^V 7/ information; questions about attitudes covered support for constitutional 

' rights and respect for others. In this report, the first three chapters 
^dej^be' results for the three major areis; Each Siapter includes "a summary 
) X -? chiiges in national perform£mce, ^mples of the items used, and data for 

\ V . ^toious population groups. Chapter four compares achievement for 13 

/ ^ ^'>- -^^siid, 17 year-olds- Overall, results of the assessment revealed that 9 year-olds 

shbwed no statistically sigmficant change in performance, and that 13 and 
IT year-olds' parformance declined. Seventeen year^ld Hispanic students 
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and students living in the West displayed smaller declines than the nation 
as a whole. Although overall performance of blacks remained belo.w that 
of whites, at age 9 blacks showed more improvement than whites. At 
age 13 black performance remained the same while white performance ^ 
declined. ' ^ . ^ ^ \ 

Elam, S., The' Nuclear Reaction/SAT Decline Connection, Phi Delta Kappan, 
November 1979. 

This article relays an interview between Elam and Ernest Stemglass. 
Stemglass traces SAT declines- by geographical area and relates fallout to 
SAT declines in ,^ch area across different periods. Stemglass' original , 
paper on this subject was presented at APA in 1979 and is cited as #220' 
in the bibliography of this review. 



Eurich,^,A., The Reading Abilities of College Students— Fifty Years Later, 
Academy News, May 19^0. 

In 1978, Eurich^replicated, at the University of Minnesota, a study he con- 
ducted in 1928 on reading tests he developed. He then compared results . 
over the 50 year interim period. His results'showed significantly poorer 
performance for the' 19'i^ group on vocabulary, reading comprehension, 
^ and reading rate. He alsoWicluded that 1978 freshmen were at least one 
grade level below the 1928 ^dents. 

Study is methodological^ weak both in t^rms of testing deyices used and 
lack of control for extraneous variables. 

^^^^ * '/ ' ' 

Examination of Ba^ic. Skills Achievement Designed to Assist Both States^ 
and liocal School Districts to Expand and Improve Their Programs in 
iBasic Skills in tjie. Elementary and Secondary Grades, 1979.. 

Twelve hundred page testimony including numerous documents submitted 
for the record on basic skills. Marvelous resource document wliifih includes 
testimony by Copperirian, Flesch, Howe, Wirtz, Graham (of NIE), Chall, 
Farr,'and Wood (Massachusetts DOE), among others. 

Farr, R., & Olshaysky, J., Is minimum competency testing the appropriate 
solution to the SAT decline? Phi Delta Kappan. April 1980. 

The authors answer no to the question they ask in the title to this article. " 
They-point out^that the SAT decline does not reflect basic competencies 
and thus it is inappropriate to prescribe minimum pomp^ency testing as a 
cure for falling SAT scores."^ They also pj)int odt that **there is no evidence 
of widespread. lack of basic literacy for (high school upper-classmen).'* 
They emphasize ^hat the NAEP results support this point-of-view in the 
areas of basic literacy skills and practical verbal tasks. They conclude by 
stating tliat basic literacy is "already at a very high level," aifi lhat 

' *If 43tate or^school system waists to unprove literacy 
levels, it does not seem that greater emphasis is 
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needed on lower level reading achievfejnent. What 
is needed is increased emphasis on higher level 
reading/thinking skills. 



*** # 83. 



-Ferguson, R., & Maxey, J., Trends in the academic performance of Ijigh 
school and coUege students (ER 109 523), 1975. 

(This study) examines ^he high school averages, ACT test scores, and first 
college semester grade point averages of high school .students who entered 
the ACT Assessment Program in 1964-65. The authors . . . foimd that 
over an eight-year period, grades awarded by high school and college faculty ' 
showed a significant increase while, at the same time, ACT test scores were 
on the decline. While noting a number of possible explanations for this 
phenomenon, they believe ... the rise in grade point averages, both at the; 
high school and college level (is d\je) to an increased leniency in the evalua- 
tion of student performance, that is, to a lowering of the academic standards,' 
arbitrary as they were, of past years. At the same time, it is (the authors) 
judgement that the decline in ACT scores is the result of a significant shift 
in the developed academic abilities of the population of students applying 
and being admitted to college. - 

The authors offer several recommendations for policies and practices in \ 
postsecondary institutions in light of their findings. \ 



# 87. Ford,.S., & Campos, S., Summary of validity data from the Admissions Tfesting 
^ogram Validity Study Service, June 1977. y 

Validity data (prediction of fijrst-year grade-point averageMor colleges - 
participating in the Admissions Testing Program Validity Siudy Service (VSS) 
and basW on studeftts entering college in 1964 through 19Tf4 are summarized 
for the following predictors: SAT- Verbal score, SAT-Matheinatical score, 
high school record, and these three predictors combined. \ 

No definite trends over time are apparent for SAT validities,mhough those . 
obtained for 1973 and 1974 were generally among the higheat observed. 
There was a downward trend in multiple correlations for all tmee predictors 
combined. Median validities for colleges having SAT-V means\Detween 450^ 
and 549 tended to be higher than those fpr colleges below 450.pr above 550. 
The great majority of colleges participa|;ing in the VSS were found to be four 
year colleges. Separate analyses of the small number of two ye^ colleges in 
the sample yielded slightly lower median validities for each of thfe predictors 
than those foimd for the total sample. • \\ 



# 92. The GED Statistical Report, 1979. ^ \\^ . 

Statistical data on 1952-1979 GED testing. Table 3; ComparatiV^ Testing 
Data, 1952-79, tracks percentage meeting GED minimum passing \ktan 
Data show Consistent decline over the period, although populatioq^ changed 
through more and yoimger examinees'. Avera^' years of education consistent 
through period. ^ • ^ \\ 



ERIC 



27 



# 94a. Glover, R., Major societahehanges in U.S. (1933-44 and I-Q-IS-TS): Con? 
textvial mapping, ^pril 1976 (revised August 1976). 

For a span of 42 years (1933-7\5) a fold-out chart displays major events and 
influences on eventis, products, and other phenomena,' suggesting their inter- 
connectedness in graphic form. iSe presentation covers occurrei^es in 
V -the following areas; political, social and cultural, economic and financial, 
legislative and legal, pepiilation and human ecology, and Idiowledge andy 
technology. The August 1976 revision is accompanied by a scenario f9r , 
theyfeod 1976-1985. 

* 

#103 Hamischf^ger, A., & Wiley, D., Achievement Test Score Decline: Do We 
Need to Worry? 1976. • ' ' 

An outstanding early source on the declining test scores. Study is w6U:\.. 

/^\^vritten and well-documented with loads of data. Should be a prime source 
for any study ^of subject. Authors document reality of decline and con- 
sider the social and educational context of achievement test score declines. 

, They evaluated most of the hypotheses for the causes of the declines and 
recommended research studies where data were lacking. A classic reference. 

#105. . Hamischfeger, A., & Wiley, D.,' Achievement Test Scores Drop. So What? 
March 1976. , ' ' 

'i 

V Primarily an excerpt fifom the earlier monograph, this article summarizes, 
the various test trends and discusses causes underlying the trends. A good 
short article, although the .full monograph lias much more information in it. 

#110.* Hess, F., Martin, M(^arker, D., & Beck,, J., An Examination of Trends ' ~ 
in Performance on ACT and SAT Examinations, 1978. 

* Outstanding source on subject, relatively short, though concise discussion 

of declines and potential causes. Som^ attempts to^ quantify percent. of 
decline to valrious. causes. Perhaps the best source for a quick, authoritative 
discussion OfW subject. 

#111. Higgins, J., etal., Assessing Mathematical Achievement, December 1979. 

This report was compiled to serve as a reference on assessments of achieve- 
ment in pre-cdUege" mathematics. Patterns of mathematical assessments are 
discussed in terms of the history and nature of assess&ents of achievement, 
the relationship between assessment and minimum competency testing, and, 
the current status of state assessment progress. Trends in mathematics 
achievement are exapained by presentation of portions of reports of the 
National Assessment of Educational Progre^ and the California Assessment, 
, grades six and twelye. ^Conclu^ns drawn from examination of these assess- 
ment data include: (1)' Although schools have been successful in teaching 
whole number computation, they have been only moderately successful 
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I ^th decimals and even less successful in .teaching computation with frac- 

/tions: (2) Students who know **how" to compute frequently do not know 
"when" or "where" to compute:^ (3) Schools must broader; their view of 4 
basic arithmetic if they expect students to use what they are taught. An . 
extensive list of references is included. 

#114. Hodkinson, H., What^s Right with Education, Phi Delta Kappan, November 1979. 

* Former head of NIE argues that things aren^ really so bad with education. 
Provides comparisoA data on international achievement in mathematics, ^ . 
. - " ' science, and reading and also* graduate admission test data from the 1977 
Conditions of Educatign. Author*s bias shows through. 

** #115. ^Holmes, B., & Wright, D., Whati)o Young Adults Know About Science? ' 
Some Results From Two NationarAssessments., February 1980. 

Data for this report come from the National Assessment of Educational 
Progress (NAEP), gathered in three science assessments of 9-, 13-, dnd 17- 
year-olds, and young adults. The assessments were conducted in 1969-70, 
1972r73, and 1976-77. This paper focuses on changes in science knowledge 
of young adults from the second assessment to the jthird assessment and 
compares performance of young adults with -that of 17-year-olds. Results 
report a uniform decline in scientific literacy among young adults and 17- \ - 
year-olds, with no difference between the two groups in their performance 
on science exercises. Observations are also noted regarding patterns of 
responses as to gender, minority groups, and social factors. 

* #120. Jackson, R., Comparison of SAT Score Trends in Selected Schools Judged 
to Have Traditional or Experimental Orientations. October 1976. 

A group of schools consi'sting largely of highly regarded schools in affluent 
suburban areas was divided into t|^o groups-those judged to have a tradi- 
tional orientation and those judged to have a more experimental orientation 
(in terms of course* and program structure). A review of the mean SAT 
scpr«s of SAT candidates from the 1966, 1969, 1973, and 1976 graduating, 
classes from ihese schools shows that (a) mean SAT scores for both school ' 
groups were substantially higher than national averages, and (b) declines in 
mean SAT scores for both groups over the period studied very nearly 
paralleled the national declines. 

• This was a small-scale pilot test using available data. Becaure of the relatively 

fallible procedures used for classifying schools and because of a general lack 
9f experimental pontrols of the effects of extraneous, factors, the results ^ 
' * cannot suppbrt^any general conclusions about thepossible relation of / 
experimentation in the schools to score decline. Because of the essentially 
negative findings of this pilot test, a more substantial study of this grbup 
of schools was not attempted. 
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* #121. Jackson, R., Correlations of SAT Scores with High School Record, 
January 1977. * 

Two sets of data are examined: '(a) correlations of SAT scores with student 
* ' "reports of class standing (collected by means of the student Descriptive 
Questionnaire) for 197f-72 through 1975-76, and (b) correlations of SAT 
scores with pleasures of high school performance for groups of students 
attending colleges participating in the College Board Validity Study Ser^e, 
for^entering classes from 1964 through 1974. The correlations of &AT 
. . scores with self-reported class-rank are virtually level over the five years 
studied* The longer-term comparisons using validity study dat^^suggest 
that the medifin correlations of -SAT verbal or mathematical scores with ^ 
high school record may have increasM somewhat 'cji^er the 11 years studied. 



5^122. Jackson, R., An Examination of Declining Numbers of High-Scoring SAT 
Candidates.^ January 1977. ^ \ ' ' ^ 



r 



* #123. 



The sharp declines in numbers of SAT candidates scoring over 6Cr07^wHicir~ 
were observed from 1969-70 through 1974-75, are examined. Several sug- 
gested explanations of this phenomenon relating to test-taking patterns are 
discussed. Speciifti^y examined are the possibilities that less SAT-to-SAT 
repetition, or less PS AT-to-S AT repetition, *or less overlap between SAT 
and ACT'among high^scoring students might account for some^Jjart of the 
observed declines. Although ideal data for testing all these explanations 
were not available, certain data in Jiand' suggest that, while changes in test- 
taking patterns may have had some effect, they probably cannot entirely 
explain the trend in question. 

Jackson, R., Mean^res for PSAT and PSAT/NMSQT Junior Candidates: 
December 20, 1976. 

* * 

Gives the numbers of students taking the preliminary Scholastic Aptitude 
Test and the Preliminary Scholastic Aptitude Test /National Merit Scholar- 
ship Qualifying Test and their, mean scores for' the period 1959 through 
1976. The implications of these results in relation to changes in the candi- 
date group are discussed. 



#124511^ Jackson, R., A Summary of SAT Score Statistics for College^Board Candi- 
dates. December 1975 (revised October 1976). ' ^ ^ " 

* SOmmaries of SAT scorQ. statistics for test candidates are presented for 
students grouped according to testing years (for 1956-57 to 1975-76) and 
-for students grouged according to h^h school classes (from 1966^7 to 
_ ' -1975-76). For recent years, mean scores for geographical ^regioi^'and for 
• students grouped according to select responses to the Studeiit Descriptive 
Questionnaire are also presented. 
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* #125. Jackson, R., Trends in Mean SAT Scores for Students from the High 
School Classes of 1971-72 and 1975-76 Belonging to Selected School 
' and College Groups. January 1977. \ . . * 

* Data ai^e presented on trends in SAT performance from the ye^ 1971-72 

j * to 1975-76. for students who (a) attended certain selected high schools for. 

' which group summary reports had been produced tlirough the College ^ 

Board summary report service, or* (t) had«their scores reported to colleges 
in groups similarly formed for summary reporting purposes.. ^AlLschoOj and 
college groups for which reports were produced both in 1971-72 and 1975-76 
' are included. The great majority of these groups exhibited declines in 
SAT sco res over , this period. With only a few exceptions" those 
showing increased in either SAT-verbal or a^T-mathematical me^i^ores 
had substantmlly smaller numbers of students in the more i;ec 

*** #147. Mayville, W., The Trouble With' Grading . . . 

This study will consider how grading systems evolved in this coSntry and 
^ ^how this evolution relates to the changing meaning of grades to the student, 
the teachir, professional and graduate schools, and society. 

The paper includes sections on: Historical Variations of Grading Formats, 
Grading Options and Uses of Grades,'and Measuring Student Achievement. 
The author surveys the various explanations for ^'grade inflation"— from 
' leniency, of grading during the Vietnam War to prevent jmale students frofii 
' ° being (kafted, to the adoption of pass-fail options.. But whatev^ tHe con- 
tributing factors, "grade inflation seems to carry witii it the accumulated 
frustration, both surtlcuktjed and suppressed, of students and their teachers 
^ ' 'over what constitutes a^lid<statement of student achievement in' any 

particular course." « , , • - 

■ — ' 

) . . The author contends that a systematic assessment- of the grading situation . 

is esserifialjTahi^bncludeis:^ ^ ~^ — — 

The belief that learning is,independent of evaluation 
has profound consequences and could be the idea 

- ^ around which future grading systems will evolve . . 

diverse student populations who, are in th^ process of 
, being educated are in danger of bling poorly served 

" oin theirMe&e for a higher education. ' . ^^ 

\ " t * ^ ^ 0 

♦ •> 

•*** #148. MelicaiyG., & Feldt, L., An Empirical Sti^dy of ttie Zajonc-Markus 

lypofliesis for Achievement Test Score Decline, AERJ. Spring 1980. 




Zajonc has proposed that , the decline in high school achievement since 
1965 c^ be explatoed by the trend from 1947 to 1962 toward larger," 
closer-spaced families. This deduction is based on the Zajonc-Markus 
theory relating child *spacing% intellectWl (development. The present 
study tegted thi^ theory with data on students in Iowa high schools. 
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Modu, C, & Stern, J., The Stability of the' SAT^Verbal icore Scale. 
September 1976. . , . . ^ 

This study was designed to assess the stability of the score scale between 
1963 and 1973 for the verbal sections of theiSAt/.In a previous study, 
scores on two old forms wete equtft^^^a 1973 form an^, through that . 
form, to the College Board scale by means of sets of item^comraon to . 
the new and old forms. This earlier study Suggested that .the S)^- scale 
had shifted upward by an average of 14 points on the verbal sections and 
17 points on the mathematical sections. k ' 

In the present study, pairs of 1963 and 1973 forms of the :S AT- Were ^ 
administered in counterbalanced order to spaced ,gamples of the same group 
with each candidate taking a 1963 and a 1973 form. The obtained scores . 
were used to plage the 1973. scores on the reporting scale u^edi for the 
1963 form. The experimentally derived scores on the. 1963 scale were 
then compared with their corresponding scores on the 1973 scal^ for candi- 
dates of th^same ability levels in order to estijnate th9 degree of scale 
shift. . " 

The findin'gs of the present Study confirm an upward scale drift which 
>gave the 1973 candidate group an average of S'^to 10 points higher than . 
they would have earned had these experimental eguating results rather 
than the qperational equating ifesults been used in reporting the 1973 
scores. The scale drift observed for the present study was not uniform 
over the extent of -the scale; it was found to increase as scores decreased 
from 600 to 200. Little-scale drift was noticed over the 10-year span 
for scores of 650 and above. Based on the Evidence from ^this study it 
would appear that the reported declines in meah SAT-VerjDal scores from 

• 1963 to 1973 are about 8 to 10 points smaller than thejr would have . 

* been had the scale been coiApletely stable. • 



Munday, L., Changing Test Scores, Especially Since 1970, March 1979. 

- — „^ . — — „ — * ft 

An authority on testing offers some "uneasy generalizations": 1) Sharp 
declines beginning in the late 1960s have 8nde4^.2) since 1970 tiiere have- 
been continured achievement gains in the lower 'grades; 3) since 1970' 
there has been a leveling off in the middle 'and upper grades with some ^ 
small declines; 4) today's achievement levels compare favorably with 
available history*; elementary children may be at a- historical highrpoint;'and 
5) today's high school achievement is about even ^th that of the e^ly 
sixties. (Author's abstract). ^ , 

This^ Article has been rebutted by Ctopperman (43) and others as overly 
positive. Recent data seem to agree with Munday 's critics. Article is 
well-written and/ documented and provides a good data source, g 



Munday, L. Declining Admission Test Scopes. 1976! 

One of the^best early sources on test ^core declines. Lots of data from 
nupierous tests. Some, like* the Canadian Tests of Basic Skills, only found 
this source. Excellent analysis and weltsu^porte^^ conclusions. Should be- 
included in all reviews of the Subject area. , 
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^* #170. Nelson, J.E., A Review of Data Available Regarding Family Income and 
Financial Aid CharacteHstics of 'Students. October 1916., 

♦ 

Greatly expanded programs of financial aid based "on need have made it 
possible for large nmnbers of low-income students td attend college. Sinca 
♦ test scores have high correlation with family income, has the increase in low- 

' income students taking the SAT produced the decline in mean scores? 

This review of available College Board candidate data, income distributions 
of entering.freshmen, percentage of age groups in college by incope levels, 
and impact of student aid on low-income enrollment shows an increased 
\ * , percentage of students coming from lower-income families before 1972, 
but no significant change since that time. \^ 

X* #188. Available Evidence on Public Attitudes Toward Education, June 1^76 (Draft). 

Questions dealing with education asked bj^^^^erican survey organizations 
since 1936 are examined, and over 60 taken to illustrate (1) changes in 
public opinion since 1960 that might have allared students' evaluations of 
college going or academic excellence, and (2) chailges p^or to 1960 that 
-might have affected the quality of preparation for college. 

Tables summarize responses, td the following broad groups of questions? ^ 
satisfaction with pubUc schools^ and personnel and with college and univer- 
gity^personnel; adequacy of homework in the public schools, of time spent 
in sdibol, of discipline in the public schools, of the public school curricu* 
• • lum, of treatment of students with diffenng ability; desirability of nation- . 
wide testing programs; and evaluation of the teaching profession * 

»• * It is found that there was a period of criticism of all levels of schoolin|^ 

^ V during the late 1950s, increasing satisfaction With schools and colleges 

) during the 1960s, and a renewed dissatisfaction that starts at the end of 

the 'sixties. It would probably be impossible to reconstruct opinion changes 
^ ' . between 1950 and 1970 in sufficient detail to relate them to changes in 
SAT scores. 

A catalog of survey questions dealing with education, other than the ques- 
tions analyzed in 'the report, is appended. 

—A supple?ien^j^||eed's paper, consisting of excerpts from the Eighth ^ 
Annual Survey af^e Public's Attitudes Toward the Public Schools. A 
project jointly conducted by The Gallup Poll and the Charles F. Kettering 
Foundation, ' Princeton, New Jersey: Public Opinion Surveys, Spring 1976. 

^ ^A significant drop in ^he public's rating ^the schools was recorded between 
1974 and 1975, but this decline leveled off between 1975 and 1^6. 
Adults perceive thp following as the major problems of the local public 
schools: lack of discipline; integration, segreation, and busing; lack of 
proper financial support; poor curriculum; lise of ^gs; difficulty of getting 
^ "good'' teachers; parents' lack of interest; size of scl\pol or classes or both; 
'\ school board pblicies; pupils' lack of interest. * 

* ' ^^nie public believes ^that the decline in national test scores in recent years 
/ • ' m^s that the quality of education is declining; it would like more attention 
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paid to basic skiUis and to discipline in the schools in order to counteract 
the perceived decline of quality in the schools. It inclines, however, to 
place blame on. parents rather than preponderantly on the schools. 



*** #189. Rever, P., & Kojaku, .L., Access^ Attrition, Test Scores and Grades of 
College Entrants and PeYsisters: 1965-1973, (ED 109 264, 1975). 

The study is based on two groups of students, one comprised of students 
wha completed the ACT Assessment Program and subsequently enrolled 
in college; and the other comprise^ of all students in the first group who 
-completed their first year of college (first-year persisters). Data collected, 
show a decline in ACT test sqores, an increase in the high school grades ' 
of college entrants, and the stability of test scores and increases in college' 
grades of freshmen completing their first year of college. 




#192[v Rimland, B., & Larson, G., The Manpower Quality Decline: An Ecologi- 
y \ . cal Perspective. November 1980. 

Outdftandiijg' source on non-traditional hypotheses causing the dedine in 
test scores. Cites data on juvenile delinquency rates, crime, hyperkenesis, 
learning disability, radioactive falfout, drug intake, food additives, etc. 
ahd tK8r relationship to incidence of test -score declines, ^cellent 
bibliography. Study requires prior approval from NPRDC to cite or quote 
, document. • * • 

/ m ^ ^ ^ 

#194.,' Rogers, V., & Baron, J'., Declining Scores: A Humanistic Explanation. 
. Phi Delta Kappan, December 1976. 

A response to EbeL's conservative view (#60). The authors believe that 
**verbal knowledge" as described by Ebel is too narrow a view of what 
• ^ education is or should be. They argue for a school experience as rich and 
rewarding for all children as the lives we hope^tKey will lead in adulthood. 
(Abstract by authors). 

* #195. Rosenthal, E., and Beaton, A.E., Annotated Bibliography. December 1, 
; ^ 1975 (revised April 28, 1976). 

More than 30 reports^d publibations pertaining to national declines in 
abilities are cited, and most.are ^innotated. The citations in this biblio- 
graphy include evidence available at the time»the panel began its work. 

* j^204. Schrader, W.B., Distribution of SAT Scores to Colleges as an Indicator 
/ of Changes in the SAT Candidate Population. September 1976. 

- 

^ This study identifies trends in the extent to which SAT candidates were 
'applying to various groups of colleges in 1960-61, 1966-67, and 19^3-74. 
> * * to 
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Colleges were grouped on the basis of: (1) the classificatidn developed by 
the Cam^e Commission on Higher^ .Education, (2) the test or tests they 
reqxiiredL applicants^ to take^ and (3^ state and region. Results provide • 
pertinent data on two main topics: (a) widening access to higher education 
and (b) acceptance of either SAT or ACT scores for admissions by some 
colleges. Although fhe interpretation of the data is complicated, especially 
by the lack of knowledge about the relation between college clioice and 
taking the SAT,. the jresults offer some reason to believe that both these 
factors resulted in changes in test taking between 1966-67 and 1971-72 
that could have contributed to the SAT score decline. They do not, 
however, provide a numOTcal estimate of the size of the effects.** 



* #206./ Schramm, W., Televisions and the Test Scores. Au^st 1976. 

There is no conclusive evidence as yet that television has been a sufficient 
cause for decline in test scores, although it may be one of several elements 
in a complex causal system. The major studies show that television view- 
ing, after the early school years, tends to be associated with lower-than- 
average achievement, although the relative extent to "which viewing affects 
achievement or unsatisfactory achievement encourages ghUdren to take 
refuge in television is not fuUy understood. Television reduces reading 
time, social interaction timej and the opportunity to practice certain 
skills necessary to academic excellence. It tends to^ reduce the average 
' level of intellectual stimulation available to a child after the age of 9 or so. 
The trend of the evidence is that television viewing pa^ttems belong to a 
group of strong variables that interact with each other and with school 
, (and, therefore, test) performance, probably with negative effect. * 



* #218. ' Stem, J., Selected Percentiles for Scholastic Aptitude Test Scores (1966- 

1967 through 1975-76): January 1977. 

This table presents- the ninetieth, seventy-fifth, fiftieth, twenty-fifth^ and " 
tenth percentile groupings for SAT Verbal and Mathematical scores • * 
reported in the 10-year period between 1966-67 and 1975-76. A testing • 
year is defined as extending from September to August; data are collecteci 
without regard to level of preparation or the number of times an individual 
student was tested during the defined yeal:. ^ • ' 

* #219 Stem, J., Table of SAT and Achievement Test Scores for Samples of 

Candidates Taking Achievement Tests 1966-67 to 1975-76. January 1977. 

The table 'provides the SA^ and the Achievement Test means and standard 
deviations for samples of candidates t^ng each of the '15 Achievement 
Tests offered 'in the Admissions Testing Program battery during the period 
1966-67 to 1975-76. The samples, drawn annually through 1971-72 and, . 
bienni$iily thereafter, provide estimated of SAT means for eafch^ Achieve- 
, ^ ment Test population. 
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A listing of hypotheses to explaiil the 'SAT scpre decline (advanced in 
letters to the. College Board, to the Advisory Panel, in ma^ine^icles, 
and in newspaper stories) is ^presented and categorized usixm^^-f^-part 
classification scherge: changes in the schools, changes in ^ciety, changes 
in the population, and problems with the teste. The first major category 
(changes in. the schools) is further broken down into hypotheses relatmg 
to curriculum, institutional policies, teachei;s, and students. The second 
major category (changes in society) lists hypotheses related to family, 
religion, civil rights, crisis of'values? national priorities, economic, labor 
movement in education, and technological charlges. 

Winter, D.G., Motivational Factors in the SAT Score Decline. October 
' 197B (Draft). ^ - ^ 4. . ^ 

The paper* examines the rese^ch evidence arid theory about three human 
. social motives that could be expected to^play some role in test performance 
\and -academic functioning: the motivesibr achievement, for affiliation, and 
for.power. It a3o consid^s the possibility of a ^'motivational overload.''^ 
Broad \2ultural trends (i.e., post-Sputnik^emphasis on academic achievement, 
revolt of the counWculture, erisis of values in Vietnam and Watergate) 
and related specific changes in.the educational system (**open," "altemafive,'* 
and^humanistic" education) may have? had direct and indirect effects on 
ipotives. Some of the hypotheses suggest that actual motive levels went up 
or down with corresponding effects on academic performance. Others 
suggest the ways in which social forces may have tihanged beliefs about the 
nature and value oi academic work, beliefs thai' may interact with relatively 
constant motive levels to produce behavior clfe|iges. 
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